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ABSTRACT 

The evaluation criteria and procedures described in 
this report were developed at the Community College of Philadelphia 
(CCP) for conducting audits of the college's programs, disciplines, 
and support services. Part I provides an introduction to the audit 
process, which involves the following steps: (1) identification of 
curricular areas to be audited; (2) creation of an audit team within 
the department or organizational unit responsible for the audit; (3) 
development of an audit plan through discussions with program 
faculty, meetings between the audit team and appropriate 
administrators, and review and approval of the plan; (4) completion 
of the audit; (5) presentation of the audit by the college staff; and 
(6) audit follow-up. This section also discusses the role of the 
audit steering committee, and the format of the evaluation report. 
Part II outlines the criteria to be used in conducting the audit, 
including curricular purposes and goals, instructional methodologies, 
enrollment and retention patterns, student outcomes, demand and need 
for the curriculum, and operating cost and efficiency. For each of 
the criteria, possible sources of data and examples of questions to 
be addressed by the audit are suggested. (EJV) 
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PREFACE 



The Audit Process at the Community College of Philadelphia 
was initiated by the Office of the Provost during the 1977-78 
year. The impetus for undertaking the audit process came from 
several sources. The Offices of Academic Affairs and 
Institutional Research had been engaged in program-related 
research for a number of years and felt that the somewhat 
disparate research efforts should be pulled together into a more 
formal program assessment. The Academic Affairs Committee of the 
Board of Trustees endorsed these efforts, and f ormalized the 
process by requesting that the results of program audits be 
submitted to them on a schedule which would result in each 
program being audited every four years. 

Also, in 1977 the State Board of Education adopted 
guidelines recommended by the Department of Education on program 
audit s . These were incorporated into Chapt er 35 , "Regulat ions 
for Community Colleges" of the Board of Education regulations. 
These guidelines provide a broad set of general criteria to be 
addressed in a program audit and include the expectation that all 
programs will be audited every five years. The procedures, 
specific criteria and audit formats are left to the discretion of 
the individual colleges. 

The audit process was expanded to include disciplines and 
support services in 1981 to insure a periodic review of 
disciplines, courses and services that are not evaluated through 
the College's program audits. As is the case for program audits, 
the results of discipline and service audits are reviewed by the 
Board of Trustees. The discipline and service audit process both 
satisfies the College's internal need for monitoring academic 
disciplines and provides institutional assessment data for the 
Middle States and other specialized accredit at ion procedures. 
The attached guidelines are to be used for the program and 
discipline audits . Separate audit procedures will be prepared 
during the Fall, 1986 term for Support Services. 

The audit criteria and procedures which follow represent a 
ma j or revision to earlier versions of the audit guidelines. They 
reflect one outcome of a project supported by the National Qenter 
of Higher Education Management Systems (NCHEMS) and the Kellogg 
Foundation. This project was focused on strengthening the audit 
process technically. But, even more importantly, the project was 
directed toward integrating the audit process into academic 
department planning and decision making • Members of the 
NCHEMS/Kellogg Project Steering Committee were Tony Bruno, 
Corrinne Caldwell , Doug Fenwi ck , Joyce Garozzo , Peggy Grip , Jane 
Grosset, David Ishizaki, Susan Mogil, Sherry Rose-Bond, Don 
Slavin, Aram Terzian, and Doug Whyte. Without their support and 
suggestions, these guidelines and procedures could not have been 
developed • 



Thomas R. Hawk 
July, 1986 



PART I 



INTRODUCTION TO THE AUDIT PROCESS 



The acadenjic audits at CCP serve two key purposes. The 
first is to provid'e a summative description of the program for 
members of the College community, including board members, 
faculty , and staff , who may not be f ami liar with the purposes and 
characteristics of the College curricula. Frequently the 
background materials used to establish a curricular offering are 
lost in College or department archives, and are not accessible to 
Board members or staff who become associated with the College 
after the program is established. As a result, the need for 
courses and programs and their goals and philosophy may not be 
known , or may be mi sunders t ood , by many College constituents • 
The summative phase provides an opportunity to re-orient staff 
and Board members to the purposes of the program and the reasons 
for its existence. Critically, it also provides an opportunity 
for the faculty and administration to re-examine their beliefs 
and assumptions regarding the curriculum and to reaffirm or alter 
historical directions that were taken in establishing the 
curriculum. 

The second purpose of the audit is to provide an opportunity 
for an in-depth look at critical issues which may be confronting 
an area of the College at the time of the audit. The audit 
allows circumstances such as changes in students 1 abilities or 
expectations, or broader environmental changes to be 
systematically examined for their implications for curricular 
change. The audit may also provide an opportunity for the 
assessment of instructional methodologies to see if improved 
student outcomes could occur through alterations to instructional 
content or methods. 

While the audit conceptually has two components, the audit 
research effort will proceed in an integrated fashion with one 
overall data collection plan which will address both the critical 
issues and summative aspects of the audit. Much of the data 
required in the summative phase will have already been collected 
and compiled by the Office of Institutional Research in its 
Performance Indicators Reports (see Appendix I) . 

Audit Process 

Successful audits do not require lengthy documents or an 
inordinate amount of staff time to complete. They do require 
careful planning and in-depth discussions prior to undertaking 
any data collection efforts to insure staff involved in the audit 
understand how to obtain needed data, and that the critical 
issues to be addressed have been fully described. To accomplish 
this, the following steps should be taken: 
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A* Identification of Curricular Areas to Be Audited 

Prior to the beginning of each academic year, the Vice 
President for Academic Affairs, in consultation with his 
staff, will identify those programs , disciplines and 
related services which will be audited during the coming 
year. The selection process will be governed by Board 
expectations for periodic program audit s and by the 
results of the Annual Performance Indicators review 
process which identifies curricular areas experiencing 
unusual change . 

B. Creation of An Audit Team Within the Department or 
Organizational Unit Responsible for the Audit 

The audit process should be a shared responsibility for 
faculty and professional staff within the curricular 
area being studied. Typically, the Department Head or 
Program Coordinator will assume a leadership role for 
insuring that the audit is complet ed . However , a 
representative group of the curriculum staff should be 
involved in the actual audit preparation. All 
curriculum staff should be involved in forming the 
critical questions and reviewing the results. 

C. Development of an Audit Plan 

Before any data collection begins, an audit plan must be 
prepared and agreed to by the appropriate Department 
Head, Division Dean and the Vice President for Academic 
Affairs. An Audit Placing Form is attached which should 
be used to facilitate the presentation of the audit 
plan . 

The steps which should be t aken to prepare the audit 
plan include : 

1. In-depth discussion by the curricular faculty on 
past and present curricular mission and goals, and 
the identification of critical issues which the 
department believes should be addressed in the 
audit . 

2. Meeting of departmental audit team with Division 
Dean, and other administrative staff if appropriate, 

•to discuss possible institutional issues to be 
incorporated in the audit process. 

3. Meet ing of audit coordinator with Audit St eering 
Committee to discuss details on the completion of 
the audit plan including: sources of available 
information, clarification of proposed critical 



ERLC 



2 



questions , identification of appropriate research 
methodologies and clarification of college-wide 
audit criteria. 



4. Presentation and Approval of Plan 

The audit t earn should complete the audit plan based 
upon the above discussions. The Office of 
Institutional Research should review the data 
collection plan to insure feasibility given 
institutional resources. The plan should then be 
reviewed by curricular staff and the Academic 
Affairs Council • Once appropriate suggestions have 
been incorporated into the plan, the plan (including 
time table for completion) is approved by the 
Department Head , Division Dean, and Vice President 
for Academic Af fairs . 

D. Completion of the Audit 

The department al audit team is responsible for 
coordinating the collection of audit data, preparing the 
audit report and presenting it to the departmental staff 
for review and suggestions. During the audit 
preparation, the departmental audit team should meet 
intermittently with the representatives of the Audit 
Steering Committee and the Division Dean to insure that 
approaches followed are efficient and consistent with 
the audit plan. Once a rough draft of the audit is 
completed, it should be presented to both the Academic 
Af fairs Co unci 1 and the department al faculty to get 
their reactions and suggestions for completing the 
audit • 

E. Presentation of the Audit 

Because the audit provides important information for 
inst itut ional planning and budgeting purposes, it is 
important that the audit be broadly reviewed by College 
staff. The audit should be formally approved by the 
departmental faculty, the Academic Affairs Council, the 
Vice President for Academic Affairs , and the President 
before being presented to the Board of Trustees for 
their consideration. 

F. Audit Follow-up 

As c result of the audit, the faculty and administrators 
responsible for an area being audited will typically 
develop an action plan for strengthening the curricular 
area. If an improvement plan is appropriate, it will 
include, in addition to proposed curricular 
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modifications, a time-table, an identification of 
individuals responsible for actions to be taken, and an 
expected budget. The Office of Academic Affairs will 
assume a leadership role to insure appropriate changes 
are initiated and their effectiveness assessed during 
the intervening years before the next audit is 
undertaken. 



Role of the Audit Steering Committee 

An Audit Steering Committee, working cooperatively with the 
Office of Institutional Research, exists to support the 
departmental audit team in the audit process. Members of the 
St ee ring Committee can meet with the faculty and appropriate 
Division Dean to assist faculty to develop an understanding of 
the audit process; can facilitate, if desired, a discussion of 
curricular goals and structure; and will suggest research 
methodologies to address the critical issues identified by the 
department. The Audit Steering Committee fill assist with the 
completion of the audit plan. Representatives of the Audit 
Steering Committee will meet periodically with the departmental 
audit t earn to review drafts of the audit at various stages to 
assist with clarity and completeness of analysis, and to insure 
that the audit conforms with institutional standards. The Audit 
Steering Committee's role will be one of facilitation and not 
evaluation. The Steering Committee will not make judgements 
related to the quality of the program. 

Format for Present ing the Audit 

The audit report should have the tone of a concise research 
report beginning with an executive summary containing all of the 
major findings and recommendations. The structure for the audit 
should be: 

I. Executive Summary 

II. De tailed Audit Findings using the Outline presented in 
Part II 

III. Discussion of critical issues and audit findings 
related to the issues 

IV. Det ailed re commend at ions for curricular change 

including proposed time table and expected budget 
implications . 



PART II 



OUTLINE OF CRITERIA TO BE USED IN CONDUCTING THE AUDIT 



The audit should provide background information and data in 

each of the following areas: 

A. Purposes and Goals 

B* Instructional Methodologies 

C. Enrollment and Retention Patterns 

D. Student Outcomes 

E . Demand and Need for Curriculum 

F. Operating Cost and Efficiency 



The sources of data and information for the audit will include: 

- departmental records and discussions; 

- Institutional Performance Indicator Dat a ; 

- data complied by the Offices of Institutional Research and 
Academic Affairs, including student survey data; 

- correspondence and records of meetings with employers, 
transfer institutions and advisory committees ; 

- institutional records, e.g., cost data available from the 
Controllers Office; 

- special studies undert aken by the department for the 
audit* 

While inevitably some special research initiatives will need to 
be undertaken for the audit, most of the effort should be focused 
on compiling and interpreting information which is already 
available. The Institutional Performance Indicators data provide 
an all- Co liege frame of reference to use in examining the 
curricular area being audited. 

The following outline explains the above criteria in more 
detail , suggests available institutional data and describes some 
examples of critical questions that might be addressed by the 
audi t • 
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A. Curricular Purposes and Goals 



The purpose of this section is to clarify the educational 
mission of the program or discipline being audited. Information 
contained in this section should include: 

1. Brief history of the curriculum at CCP and description 
of current curriculum; 

2. Current goals; 

3. Relationship of the curriculum to the overall 
educational program of the College. 



Sources of Information 

- Department al dialogues 

- Departmental records 

- Minutes of College-wide Curri culum Committee and General 
Education Task Forces 

- Records of Instructional Development and Academic Affairs 
Offices 



Examples of Critical Questions 

1. Do the program or discipline offerings reflect a commitment 
to the College's general education goals and stated 
institutional values? 

2. Is the College catalogue representative of the current 
thinking of departmental faculty about the appropriate nature 
of the curriculum? 

3. Are course goals and objectives consistent for the various 
faculty members teaching a given course? 



B • In st ruction al Methodologies 

This section should describe the approaches used by the 
department to help students achieve curriculum goals* 
Departmental resources (e.g., laboratories, specialized teaching 
materials ) should be described, as should faculty credentials and 
backgrounds • ■ 

The extent to which the curriculum supports or depends upon 
other areas of the College should be r -.plained. Usage of 
external resources, e.g., clinical facilities, guest lecturers, 
work experiences, should be described. 
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Sources of Information 



- Departmental records 

- Records of Instructional Development and Academic Affairs 
Offices .. % 

Examples of Critical Questions 

1. Are there alternative teaching strategies that might be 
employed to improve student performance? 

2. Are students receiving a work experience in clinical settings 
that is consistent with curricular goals? 

3. Is a laboratory experience essential to the accomplishment of 
course goals? 

4. Should students receive more opportunities to apply computer 
techniques to the course content? 

5. Do some faculty members appear to have greater success in 
accomplishing curricular goals? Why? 



C . Enrollment and Retention Patterns 

The purpose of this section is to describe enrollment and 
retention patterns in the program and to highlight any changes 
which may have implications for the outcomes of the program* In 
order to understand trends, at least fo^r years of data should be 
collected for this section. 

Where appropriate, the data for the curricular area should 
be contrasted to institutional patterns. Unless specific 
critical issues require a special research effort, all of the 
data for this section will be available from standard 
institutional sources. 



Sources of Information 

- Comparable Performance Indicator Reports 
(see Section A and B of Appendix I) 

- Office of Institutional Research 

- Scheduler f s Office 
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Examples of Critical Questions 

1. Why are enrollments in the curriculum by race or sex not 
consistent with overall institutional enrollment patterns? 

2. Why are enrollments in second-level courses declining? 

3. Why is the program failing to achieve its new student quotas? 

4. Why do students leave the program without graduating? 

5. What are course -selection pat tern implications of a growing 
number of part-time and evening students? 

6. Do admission standards appear to discriminate against a 
particular age, race or sex group? 



D. Student Outcomes , 

This part of the audit is the most time-consuming, and in 
many respects, the most important part of the audit. Two types 
of information will typically be included in this section: 
statistical data detailing student academic performance while at 
the College and after in employment and transfer experiences, and 
qualitative assessment data compiled by asking former or current 
students to assess the effectiveness of the College in helping 
them to achieve their educational goals. In addition, a wide 
range of specialized rsearch initiatives are possible to measure 
cognitive and non-cognitive gains in either major or general 
education courses. 

Examples of some standard statistical measures that may be 
reported in this section include: 

- Student overall academic performance in program courses 

- Student performance on standardized and certifying exams 

- Grade distributions 

- Number graduating in each of the last (four) years 

- Number leaving program in good academic standing without 
graduating in the last four years, broken into the 
following categories (if possible): 

,lo Number accomplishing educational goals at CCP 

2. Number changing educational goals, e.g., 
transfer to another program 

3. Number leaving within accomplishing educational 
goals 



8 



- Number of students dismissed f rom program and dropped f rom 
CCP for poor scholarship or poor progress in each of the 
last four years 

- Post -CCP activities reported separately for graduates and 
no n -graduates in each of the last four years in the 
following areas: 

1* Estimated number employed full-time 

2 . Estimated number transferring 

3. Estimated number unemployed and looking for work 

- Average credits lost when transfer ring 

- Percentage of students expressing satisfaction with 
transfer preparation, employment skills preparation, etc. 



Sources of Information 

- Office of Institutional Research 

- Graduating and Non-Returning Student Surveys 

- Grade Distribution Reports 

- Comparable Performance Indicator Da\.a (Appendix I, Sections B 
and C) 

- Research studies focused on specialized outcomes issues 



Examples of Critical Questions 

1. Are students adequately prepared to do an effective job 
search in their field of study? 

2* Are students having difficulty in upper-level major courses 
after transferring from CCP? 

3» Could students 1 successes in transferring or finding 
employment be enhanced by restructuring the curricula r 
requirements? 

4* Are students consistently being prepared in first-level 
courses to meet the expectations of second-level courses? 

5. What are the characteristics of students who withdraw or fail 
from the curriculum? Can these withdrawals or failures be 
prevent ed? 
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E. Demand and Need for the Curriculum 



This section is intended to describe the continuing need for 
the curriculum based upon student interest, employment 
opportunities, employer attitudes toward the program, and 
transfer opp jr tuni t ie s . Information collected for this section 
may be derived from published manpower studies as well as from 
institutional data and institutional surveys. 

Student interest in the curricular area can be demonstrated 
by highlighting data related to enrollments provided in Part C. 
Employment opportunity can be explained in part by the employment 
outcomes of former students including related salary data. 
Advisory Committee members may help to establish employment 
opportunities for specific firms or industries. Frequently, 
professional organizations will have published studies 
documenting opportunities in a particular employment field. 
Formal and informal transfer agreements can serve as evidence of 
the desire for four-year schools to enroll former CCP students. 
Special scholarship opportunities may also help document transfer 
institution interest. The extent to which other competing 
colleges are providing similar training to meet documented 
employment openings can help to explain need for a program. 



Sources of Information 

Academic Affairs Office files 
Student and employer surveys 
Library literature searches 
Office of Institutional Research 
Departmental records (prior needs surveys) 



Examples of Critical Questions 

1. Will there continue to be sufficient labor market openings to 
justify operation of the program at the current level of 
enrollments? 

2. Do graduates have appropriate skills to compete successfully 
in the job market? 

3. Are lack of employment or transfer opportunities a reason for 
high course/program withdrawals? 

4. What adjustments to the program requirements would facilitate 
student transfer? 
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F* Operating Costs and Efficiencies 

The purpose of this section is to show how well the 
department uses available resources to produce the student 
outcomes described earlier in the audit. The key issue for this 
section is whether the same level of outcomes could have been 
obtained with a smaller commitment of institutional resources , or 
conversely could a higher level of outcomes been obtained with 
the same commitment of institutional resources* 

Most or all of the information needed for this section is 
available from standard institutional reports. Staff responsible 
for curricular areas having relatively high unit costs and low 
productivity, as measured by faculty teaching loads and class 
Liizes, should explain, where appropriate, constraints such as 
clinical arrangments and accreditation requirements which produce 
the higher costs associated with the curriculum* 



Sources of Information 

Office of Controller - Cost Studies 
Office of the Scheduler 

Performance Indicator Reports (see Part E, Appendix I) 
Examples of Critical Questions 

1* Can average class sizes be increased without impairing needed 
student curricular options? 

2. Are there ways that operating costs can be reduced without 
impairing student outcomes? 
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